Multi-layer opto-electronic neural network

ABSTRACT

A pattern recognition apparatus and a method for operating same. The apparatus includes a volume holographic medium (4) having a plurality of Fourier-space volume holograms representing pattern templates stored within. The apparatus further includes a spatial light modulator (1) and a phase encoder (2). The phase encoder has an output optically coupled to the medium by a first Fourier transform lens (3). The spatial light modulator spatially modulates a spatially uniform laser beam (7) in accordance with an unknown pattern. The two-dimensional phase encoder causes the spatially modulated laser beam to be spatially distributed prior to application to the medium. The apparatus also includes a detector (6, 11) having an input optically coupled by a second Fourier transform lens (5) means to an angular spectrum of plane waves generated by the medium in response to the output of the spatial modulator, phase encoder, and first Fourier lens. The detector detects plane waves that correspond to vector inner products generated within medium (4) in response to the unknown pattern. The apparatus further contains a means (12) for nonlinearly processing the output of detector (6, 11) and a means (13) by which the output of nonlinear processing means (12) may be temporarily stored.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This patent application is a continuation-in-part of U.S. patentapplication Ser. No. 07/811,589, filed Dec. 20, 1991, entitled"Volume-Holographic Inner Product Processor" now U.S. Pat. No.5,235,439, which is a continuation-in-part of U.S. patent applicationSer. No. 07/562,294 filed 3 Aug. 1990, entitled "Continuous-Time OpticalNeural Network,"by H. M. Stoll et al., now U.S. Pat. No. 5,099,434,which is a continuation-in-part of Ser. No. 07/220,769, filed Jul. 18,1988, entitled "Optical Neural Network", by H. M. Stoll et al., nowabandoned each of which is assigned to the assignee of record herein.

BACKGROUND OF THE INVENTION

Multi-layer neural networks may be used to classify patterns. Thesenetworks typically consist of layers of nonlinear processing elements(or "neurons") arranged in a highly interconnected hierarchy. Eachneuron within the top layer of the network hierarchy accepts as input aweighted sum over all of the resolution elements of the pattern to beclassified. Each of these sums is then nonlinearly processed by eachtop-layer neuron and outputted to the second layer of the network, inwhich each neuron accepts as input a weighted sum over all neuraloutputs of the first layer. This process continues until the output, orclassification, layer of the network is reached. The outputs of thislayer are then interpreted as the desired classification results.Typically, no more than two or three layers are required to achievepattern classification and typically the number of neurons in each layerdecreases as the classification layer is approached. The network istrained to classify patterns by pre-selecting the weights thatinterconnect the various layers. A good theoretical description ofmulti-layer neural networks may be found in Parallel DistributedProcessing: Explorations in the Microstructure of Cognition, Vol. 1:Foundations, by D. E. Rumelhart and J. L. McClelland (MIT Press, 1986).

Mathematically, the functioning of a single layer of a multi-layerneural network may be described as follows:

    g[R.sup.(i)σ(i) ]=g[ƒ.sup.(i) ]=σ.sup.(i+1) ; i=1,2, . . . , N,                                                  (1)

where the pattern vector σ.sup.(i) is the input to layer "i"; the matrixR.sup.(i) represents the neuron input weights; N is the number ofnetwork layers; and g/ [·] is a nonlinear vector function which operatesidentically on each element of ƒ.sup.(i). Typically, g/ [·] operates oneach element "k" of ƒ.sup.(i) as indicated in FIG. 1. The particularnonlinear transfer function illustrated in FIG. 1 has what is commonlyreferred to as a sigmoidal shape, with adjustable threshold ("a") andsaturation ("b") points.

FIG. 2 shows an illustrative example of a three-layer neural networkconsisting of a three-resolution-element input pattern with two outputclasses. Each layer consists of a fully interconnected set of weightsconnecting the input to the summers. The output from the summers is fedthrough a nonlinearity, which completes the processing for that layer.The output from one layer serves as input to the next layer.

Pattern classification problems in which input patterns aretwo-dimensional images typically require two-layer neural networks whichmay contain as many as 10² classification-layer neurons and 10³input-layer neurons. For a 10⁴ -pixel image and fully interconnectedlayers, R.sup.(1) becomes a 10⁴ ×10³ -element matrix and R.sup.(2) a 10³×10² -element matrix. Real-time (˜10³ seconds) classification of unknownimages therefore requires on the order of twenty billion operations persecond [=2×(10⁷ +10⁵)×10³ ]. Existing, all-digital electronic computerscapable of such throughput occupy many cubic feet of volume and consumethousands of watts of power.

Optical devices in which the matrices R.sup.(i) may be stored in theform of two-dimensional Fourier-space holograms include those describedby: D. Gabor in "Character Recognition by Holography" in Nature, 208,p.422 (1965); J. T. LaMacchia and D. L. White in "Coded MultipleExposure Holograms", Applied Optics, 7, p.91 (1968); J. R. Leger and S.H. Lee in "Hybrid Optical Processor for Pattern Recognition andClassification Using a Generalized Set of Pattern Functions", AppliedOptics, 21, p.274 (1982); and D. A. Gregory and H. K. Liu in"Large-Memory Real-Time Multi-channel Multiplexed Pattern Recognition",Applied Optics, 23, p.4560 (1984). Additionally, in a paper by T.Jannson, H. M. Stoll, and C. Karaguleff ("The interconnectability ofneuro-optic processors", Proceedings of the International Society forOptical Engineering, Vol. 698, p. 157 (1986)), there is described, onpage 162, an optical volume-holographic architecture for computingmatrix-vector products. This disclosure is, however, in the context ofproviding interconnects for an all-optical, recurrent (feedback)-typeneural network.

It is one object of this invention to provide a method and apparatusthat employs a three-dimensional volume holographic medium in whichmulti-layer, opto-electronic neural network interconnects are stored andused to multiply pattern vectors.

It is another object of this invention to provide nonlinear processingmeans by which the intermediate and output pattern vectors computedwithin a multi-layer, opto-electronic neural network may be acted upon.

It is a further object of this invention to provide a compact(potentially less than 200 cubic inches), low-power (potentially lessthan 10 watts of prime electrical power), multi-layer, opto-electronicneural network capable of executing at least 2×10¹⁰ (twenty billion)arithmetic operations per second.

SUMMARY OF THE INVENTION

In accordance with the invention, a multi-layer neural network acceptsas input a pattern vector σ.sup.(1) and returns as output aclassification vector σ.sup.(N). All σ.sup.(i) (i=1,2, . . . , N-1) areassigned pre-determined portions of the network input transducer, whichconsists of a two-dimensional spatial light modulator (SLM) mounted ontoor in close proximity to a two-dimensional phase encoder (diffuser)having the same transverse dimensions. Neural network weight matrices,R.sup.(i) , are stored in the form of three-dimensional, Fourier-spaceholograms, with each hologram corresponding to a single row of anindividual R.sup.(i). These rows are referred to hereafter as weightvectors. All weight vectors of a given R.sup.(i) are accessedsimultaneously (in parallel) by illuminating the volume holograms withthe Fourier transform of the pattern or input vector, σ.sup.(i), whichis to be weighted (or multiplied) by the R.sup.(i) in question. Elementsof the product vectors ƒ.sup.(i) (inner products between the weightvectors and σ.sup.(i) are determined by measuring the properties of thelight radiated by the volume holograms: the angle of each of the lightrays radiated indexes the element (i.e., indicates which weight vectoris being multiplied or dotted onto the input vector σ.sup.(i)) while theamplitude of each of the light rays radiated is proportional to thesquare of the magnitude of the indexed inner product. Nonlinearprocessing of the individual elements of the product vector, ƒ.sup.(i),is accomplished by focusing each radiated light ray onto a detectorarray element and further processing the detector array output using,for example, an electronic look-up table or a saturable electronicamplifier with adjustable thresholding and saturation points. Thenon-linearly processed σ.sup.(i) (equal to σ.sup.(i+1) is thentemporarily held in an electronic buffer memory prior to being loadedinto that portion of the SLM allocated to layer "i+1". This procedure iscontinued until σ.sup.(N) is produced.

In addition to providing computational throughputs far in excess ofavailable or projected all-digital electronic neural networks, thepresent invention enables more weight-vector information (the product ofweight-vector count and weight-vector size) to be accessed in parallelthan do the aforementioned devices, wherein weight vectors are stored inthe form of two-dimensional Fourier-space holograms. The ratio ofstorage capacities (storage capacity of a three-dimensional-hologramdevice divided by the storage capacity of a two-dimensional-hologramdevice) is equal to the maximum (linear) space-bandwidth product, ornumber of linearly ordered resolution elements, that can be achieved inan optical system. The latter number is typically on the order of 3,000.

More specifically, the invention provides a pattern classificationapparatus and a method for operating same. The apparatus includes avolume-holographic medium having a plurality of Fourier-space volumeholograms representing stored weight vectors. The apparatus furtherincludes a spatial light modulator and a phase encoder. The phaseencoder has an output optically coupled to the volume-holographic mediumby a first Fourier transform lens. The spatial light modulator spatiallymodulates a spatially uniform laser beam in accordance with an unknownpattern which has been loaded into the spatial light modulator. Thetwo-dimensional phase encoder causes the spatially modulated laser beamto be spatially distributed prior to application to thevolume-holographic medium. The apparatus also includes a detector havingan input optically coupled by a second Fourier transform lens means toan angular spectrum of plane waves generated by the volume-holographicmedium in response to the output of the spatial modulator, phaseencoders and first Fourier lens. The detector detects focused planewaves that correspond to vector inner products generated within thevolume-holographic medium in response to the unknown pattern vector. Theapparatus further includes a nonlinear electronic device for seriallyprocessing the detected inner products, means for temporarily storingthe nonlinearly processed inner products, and means for feeding thenonlinearly processed inner products back into the spatial lightmodulator.

BRIEF DESCRIPTION OF THE DRAWINGS

The above set forth and other features of the invention are made moreapparent in the ensuing Detailed Description of the Invention, when readin conjunction with the attached drawings, wherein:

FIG. 1 depicts nonlinear processing of element "k" of the neural inputvector, [ƒ.sup.(i) ]_(k), to produce a corresponding element of theneural output vector, [σ.sup.(i+1) ]_(k). "a" and "b" are thresholdingand saturation points, respectively, of the resulting sigmoidal neuraltransfer function.

FIG. 2 schematically depicts an example of a three-layer neural network.

FIGS. 3 is a perspective drawing of a multi-layer opto-electronic neuralnetwork (MLOENN) of the invention;

FIG. 4 is a perspective drawing of the MLOENN illustrating the storageof weight vectors ν.sup.(k,i), (i=1,2, . . . , N-1), of a sequence of Kweight vectors (k=1,2, . . . , K);

FIG. 5 depicts the spatial light modulator geometry used to store theweight vectors ν.sup.(k,i), i=1,2, . . . , (N-1).

FIGS. 6a, 6b, and 6c illustrates the spatial light modulator geometryused to sequentially process patterns σ.sup.(N-1) (the input pattern tothe neural network) through σ.sup.(N) (the input to the Nth layer of anN-layer neural network, with FIG. 6a indicating the first loading ofσ.sup.(1) into the same portion of the SLM used to load the ν.sup.(k,1)(k=1,2, . . . , K), with FIG. 6b showing the subsequent loading ofσ.sup.(2) into the same portion of the SLM used to load the ν.sup.(k,2)(k=1,2, . . . , K), and FIG. 6c showing the loading process as continueduntil σ.sup.(N-1) is loaded into the same portion of the SLM used toload the ν.sup.(K,N-1) (k=1,2, . . . , K).

FIG. 7 is a plan view of the MLOENN illustrating the generationsdetection, and nonlinear processing of a single vector inner product,ƒ_(k).sup.(i) ; and

FIG. 8 is a perspective drawing of a further embodiment of an MLOENNillustrating the storage of a single weight vector ν.sup.(k,i) (i=1,2, .. . , N-1) of a sequence of weight vectors (k=1,2, . . . , K), thisembodiment employing a reference plane-wave laser beam that is scannedin two dimensions in conjunction with a two-dimensional detector array.

DETAILED DESCRIPTION OF THE INVENTION

The Multi-Layer Opto-Electronic Neural Network (MLOENN) classifiespatterns by repeatedly executing the algorithm described by Equation(1). This algorithm consists of: (1) calculating the matrix-vectorproduct R.sup.(i) σ(i) to yield ƒ.sup.(i) and (2) nonlinearly processingƒ.sup.(i) to yield σ.sup.(+1) which then becomes the input to the nextnetwork layer. The network is virtual in the sense that only one layeractually "exists" at a time: intermediate results (i.e., σ.sup.(i) fori> 1) are temporarily stored in a buffer memory prior to being fed backinto the MLOENN for further processing.

The MLOENN calculates R.sup.(i)σ(i) by computing, in parallel, the innerproducts between the rows of R.sup.(i) (hereafter referred to as weightvectors) and σ.sup.(i). Mathematically, the MLOENN calculates:

    ν.sup.(k,i) ·σ.sup.(i) =ƒ.sub.k.sup.(i) ;k=1,2, . . . , K,                                                (2)

where ν.sup.(k,i) is the (transpose) of the k^(th) row of R.sup.(i),ƒ_(k).sup.(i) is the k^(th) element of ƒ.sup.(i), and K is the number ofrows of R.sup.(i). σ.sup.(i) is a lexicographic representation of theinput pattern σ.sup.(i) (or intermediate neural network result beingprocessed), wherein each resolution element or pixel of σ.sup.(i)corresponds to a specific element of σ.sup.(i).

The MLOENN nonlinearly processes the ƒ_(k).sup.(i) by serially passingthe latter through any electronic device with an appropriate, nonlineartransfer function (as illustrated by the example shown in FIG. 1).

A perspective illustration of the MLOENN is shown in FIG. 3. The MLOENNincludes a two-dimensional spatial light modulator (SLM) 1, atwo-dimensional phase encoder 2, a first Fourier transform lens 3, amedium 4 in which volume holograms are stored, a second Fouriertransform lens 5, a linear detector array 6, a nonlinear processingdevice 12, a buffer memory 13, Rays 9 represent plane waves generatedwithin medium 4 (having amplitudes ƒ_(k).sup.(i), k=1,2, . . . , K) andrays 10 represent waves (also having amplitudes ƒ_(k).sup.(i), k=1,2, .. . , K) convergent (i.e., focused) on detector 6. With reference toEquation (1), σ.sup.(i+1) represents the input pattern to be classified,g/ [ƒ(N)] represents the ultimate classification result, and σ.sup.(i+1)represents an intermediate layer (i=1,2, . . . , N-1) pattern vector.Nonlinear processing device 12 processes each of the ƒ_(k).sup.(i)sequentially to generate (also sequentially) the σ_(k).sup.(i+1)according to Equation (1). The σ_(k).sup.(i+1) are then held temporarilyin buffer memory 13 prior to being either fed back into the SLM as inputto another network layer or extracted as the ultimate classificationresult.

In FIG. 3 SLM 1 includes means for electronically inputting a weightvector or pattern. By way of example only, SLM 1 may be comprised of aliquid crystal (LC) projection display device having a plurality ofpixels that are modified in response to input from, for examples adigital computer. The use of a LC projection display device enables anew input pattern or weight vector to be stored within medium 4 every,for example, 1/30th of a second. Any one of a number of spatial lightmodulator types may be employed. These include ferroelectric liquidcrystal, twisted nematic liquid crystal, silicon membrane (deformablemirror), and magneto-optic types. In other embodiments of the invention,SLM 1 may be simply a transparent substrate having a pattern formedthereon. Two-dimensional phase encoder 2 causes the optical signal thatpasses through SLM 1 to be spatially distributed prior to application tomedium 4. This function may be accomplished by constructing phaseencoder 2 from a transparent substrate, such as glass, and providing anetched random pattern on a surface of the substrate. The lineardimension of the smallest feature of the random pattern defines thecoherence length of phase encoder 2. The significance of the coherencelength of the phase encoder is discussed below. Fourier transform lenses3 and 5 are typically spherical lenses. A presently preferred volumehologram medium 4 is comprised of iron-doped lithium niobate (LiNbO₃:Fe). Representative dimensions of the active volume of medium 4 are onecentimeter on a side. Holograms may be "permanently" fixed by heatingthe LiNbO₃ :Fe to approximately 160° for approximately twenty-fiveminutes (see, for example, D. L. Staebler, W. J. Burk, W. Phillips, andJ. J. Amodei in "Multiple storage and exposure of fixed holograms inFe-doped LiNbO₃ ", Applied Physics Letters, Vol. 26, p.182 (1975)).Holograms fixed in such a manner are estimated to have a half-life ofapproximately 100,000 years at room temperature. Other suitable volumehologram media includes by example, strontium barium niobate (SrBaNbO₃),photorefractive photopolymers, and photochemical photopolymers. Lineardetector array 6 may be, for example, a charge-coupled device (CCD), aself-scanned diode array, a Schottky diode array, a pyroelectric devicearray, or other device capable of converting optical photons into anelectronic voltage or current. Linear detector array 6 has a resolution,or number of photoresponsive elements, equal to the number of templatesstored within medium 4. Nonlinear processor 12 may be any electronicdevice with an appropriately shaped transfer function. Examples includedigital electronic look-up tables and saturable electronic amplifiers.Buffer memory 13 may be any digital electronic memory. The output ofbuffer memory 13 may be fed back to SLM I for input to the next layer ofthe neural network by any suitable electrical connection well-known inthe art.

FIG. 4 illustrates the storage of weight vectors within the medium 4.Weight vectors are stored within medium 4 in the following manner:

1. weight vectors ν.sup.(k,i) (i=1,2, . . . , N-1) are loaded into SLM 1using a predetermined and fixed lexicographic ordering scheme;

2. SLM 1 spatially modulates a spatially uniform, plane-wave laser beam7;

3. phase encoder 2 multiplies the light pattern transmitted by SLM 1 bya random, two-dimensional phase encoding function;

4. first Fourier transform lens 3 (which is positioned one focal length(ƒ₁) from phase encoder 2 and one focal length (ƒ₁) from the midpoint ofmedium 4) generates (at approximately the midpoint of medium 4) theFourier transform of the light pattern transmitted by phase encoder 2;

5. simultaneously with step (4), reference plane-wave laser beam 8(which is temporally coherent with plane-wave laser beam 7) illuminatesmedium 4 at angle ψ_(k) to the z₂ -axis of medium 4 and within the x₂-z₂ plane; and

6. weight vector hologram V.sup.(k) forms within medium 4.

7. This procedure is repeated for k=1,2, . . . , K until allFourier-space holograms V.sup.(k) (k=1,2. . . , K) have been storedwithin medium 4.

As employed herein, a predetermined and fixed lexicographic orderingscheme is intended to mean that weight vectors are presented to thesystem in a consistent manner. For example, if the weight vector isderived from a television camera having a plurality of scanlines, thescanlines are input in the same order for each weight vector. Thescanlines need not be input sequentially, so long as they are inputconsistently.

FIG. 5 illustrates the SLM partitioning geometry used to store theweight vectors of an N-layer neural network. Since the weight vectorsare, as indicated earlier, pre-selected, the k^(th) rows of all Ninterconnect matrices R.sup.(i) (i=1,2, . . . , N-1) may be storedsimultaneously. The SLM geometry shown in FIG. 5 is capable of storingN-1 interconnect matrices, each having a maximum of K rows. Matriceshaving fewer than K rows may be stored by simply blocking (i.e.,electronically setting to zero) the appropriate portions of the SLM.

FIGS. 6a, 6b, and 6c illustrate the SLM geometry used to sequentiallyprocess patterns σ.sup.(1) through σ.sup.(N-1). σ.sup.(1) (the inputpattern to be classified) is first loaded into the SLM (all otherregions of the SLM are electronically set to zero in order to blockplane-wave laser beam 7) as shown in FIG. 6a. σ.sup.(2) is then fetchedfrom temporary memory 13 and loaded into the SLM as shown in FIG. 6b.The same procedure is followed until σ.sup.(N-1) has been loaded asshown in FIG. 6c, at which time memory 13 contains the desired patternclassification results. The geometry shown in FIGS. 6a, 6b, and 6ccorresponds exactly to the geometry shown in FIG. 5, i.e., ν.sup.(k,1)occupies the exact same physical portion of the SLM as σ.sup.(i).

Plane-wave laser beam 7 may originate from, for example, an argon-ionlaser having a wavelength of 4875 Å. The reference plane-wave laser beam8 originates from the same source. It is also within the scope of theinvention to maintain medium 4, if comprised of iron-doped lithiumniobate, at a temperature of approximately 130° C. while the weightvectors are being inputted. This results in a simultaneous storing andfixing of the weight vectors. For this case, some shrinkage of medium 4occurs when same is cooled and plane-wave laser beam 7 is required tohave a slightly shorter wavelength so as to compensate for the shrinkageof the material when applying an unknown pattern to the MLOENN.

During the storage of weight vectors within medium 4, phase encoder 2beneficially diffuses or spreads out the light energy so that the energyis uniformly distributed throughout the volume of medium 4. If phaseencoder 2 were not used, the light energy from successive weight vectorswould be focused to within a small region within the volume of medium 4.This would result in a reduction in storage capacity and an increase inoptical crosstalk.

Also during the storage of weight vectors, the reference laser beam isscanned through a plane of medium 4. As an example, reference plane-wavelaser beam 8 may be scanned through plus or minus five degrees,referenced to the center of the medium 4, in 0.01 degree increments.That is, after a weight vector is stored, reference plane-wave laserbeam 8 is shifted by 0.01 degrees before the storage of a next weightvector.

FIG. 3 illustrates the generation of vector inner products ƒ_(k).sup.(i)(rays 9 and 10 which occurs in the following manner:

1. pattern vector σ.sup.(i) is loaded into SLM 1 using the samepre-determined and fixed lexicographic ordering scheme used to load theν.sup.(k,i) ;

2. SLM 1 spatially modulates plane-wave laser beam 7;

3. phase encoder 2 multiplies the light pattern transmitted by SLM 1 bya random, two-dimensional phase encoding function;

4. first Fourier transform lens 3 generates (at approximately themidpoint of medium 4) the Fourier transform of the light patterntransmitted by phase encoder 2;

5. volume hologram medium 4 generates an angular spectrum of plane waves9 with amplitudes proportional to ν.sup.(k,i) ·σ.sup.(i) (k=1,2. . . ,K) and propagation angles of ψ_(k) (k=1,2. . . , K) which are identicalto reference plane-wave angles ψ_(k) ;

6. second Fourier transform lens 5 located one focal length (ƒ₂) fromthe midpoint of medium 4, focuses each plane wave (of the angularspectrum of plane waves) generated within volume hologram medium 4 ontolinear detector array 6 located one focal length (ƒ₂) from secondFourier transform lens 5;

7. inner products ƒ_(k).sup.(i) (corresponding to focused plane waves 10emergent from second Fourier transform lens 5) are read out of detectorarray 6;

8. the output of the detector array is serially processed by nonlinearprocessor 12;

9. the output of nonlinear processor 12 is temporarily stored withinbuffer memory 13; and

10. the contents of buffer memory 13 are either read out and interpretedas classification results or fed back into SLM 1 for further,multi-layer network processing.

The above-described determination of ƒ_(k) (for the sake of clarity, weherafter omit the superscript "i" denoting network layer) may beunderstood in greater detail by considering the electric fielddistributions which, under appropriate conditions, exist at variouspoints within the MLOENN. Accordingly, referring to FIG. 7, for anelectric field distribution incident on volume hologram medium 4 givenby

    E.sup.(1) =F{σ·e.sup.iβ},              (3)

where σ is the two-dimensional electric field distribution whichcorresponds to σ; β is the two-dimensional phase encoding functioncharacteristic of phase encoder 2 (see, for example, C. N. Kurtz in "Thetransmittance characteristics of surface diffusers and the design ofnearly band-limited binary diffusers", Journal of the Optical Society ofAmerica, Vol. 62, p. 982 (1972)); and F{·} denotes Fourier transform;and for a refractive index distribution within volume hologram mediumproportional to ##EQU1## where A^(k) is the amplitude of referenceplane-wave laser beam 8 associated with weight vector ν^(k), (·)*denotes complex conjugate, and ν^(k) is the two-dimensional fielddistribution given by

    ν.sup.k =F{ν.sup.k ·e.sup.iβ };        (5)

the electric field distribution within the plane of linear detectorarray 6 is given ##EQU2## where α^(k) is the inverse Fourier transformof A^(k), "*" denotes convolution, and " " denotes correlation.

Spatial filtering of E.sup.(2) within the plane of detector array 6 (thecorrelation plane) is performed both within and perpendicular to theplane of FIG. 7. The inner product is detected in the form of lightenergy incident on detector array 6 a distance x_(k) from the center ofarray 6 (the common optical axis of lens 5 and holographic storagemedium 4).

In-plane spatial filtering occurs as a natural result of Braggselectivity within the volume hologram medium 4 (see, for example, T.Jannson, H. M. Stoll, and C. Karaguleff in "The interconnectability ofneuro-optic processors", Proceedings of the International Society forOptical Engineering, Vol. 698, p.157 (1986)). Spatial filteringperpendicular to the plane of the processor occurs as a result of phaseencoder 2's autocorrelation function being much narrower (˜ten times)than either the autocorrelation function of σ or any of theautocorrelation functions of the ν^(k). These spatial filtering effectsyield for the field distribution within the correlation plane: ##EQU3##where the coherence length of phase encoder 2 is assumed to besignificantly smaller than (e.g., less than 10% as large as) thesmallest linear dimension of resolution element of either a or any ofthe ν^(k). The double integral in Equation (7) is taken over thecorrelation plane. E.sup.(3) may, following lexicographic ordering, bere-written as: ##EQU4## where, for reference plane-wave laser beamsA^(k),

    α.sub.k δ(x-x.sub.k);                          (9)

δ(·) is the dirac delta function; the x-dimension lies both within thecorrelation plane and within the plane of the holographic inner productprocessor; and [·,·] denotes vector inner product. Field E.sup.(3)represents the inner product of σ with each of the weight vectors ν^(k),which is the desired result.

Although described in the context of a bulk right-angle geometry systemit should be realized that the hologram geometry may be provided insteadin a transmission or a reflection (Lippmann) geometry.

Also, although the invention has been described as using a linear arrayof detectors 6 disposed along the correlation plane it should berealized that, as illustrated in FIG. 8, a two-dimensional detectorarray 11 may be employed for a system that scans, during weight vectorstorage, reference laser beam 8 in two dimensions. The two-dimensionaldetector array 11 may then be a staring type array. In this case,fractal storage geometry considerations are employed to select referencelaser beam 8 angles, ψ_(k) and Φ_(k), so as to avoid crosstalk withinmedium 4.

This invention has been described in conjunction with the illustrativeembodiments enumerated above. It will be evident to those skilled in theart that various alterations, modifications, and variations may be madeto the illustrative embodiments without departing from the spirit andscope of the claims appended hereto. For these reasons, this inventionis not to be construed as being limited to only the illustrativeembodiments, but should only be construed by reference to the appendedclaims.

What is claimed is:
 1. A multi-layer opto-electronic neural network(MLOENN) pattern classification apparatus, comprising:a volumeholographic medium having a plurality of Fourier-space volume hologramsrepresenting neural network weight vectors stored within; means, havingan output optically coupled to said medium by a first Fourier transformlens means, for spatially modulating a spatially uniform laser beam inaccordance with an unknown pattern; means, having an input opticallycoupled by a second Fourier transform lens means to an angular spectrumof plane waves generated by said medium in response to the output ofsaid spatial modulating means, for detecting plane waves that correspondto vector inner products generated within said medium in response to theunknown pattern; means by which the output of said detecting means arenonlinearly processed in a serial manner; means by which saidnonlinearly processed output is temporarily stored; and means by whichthe output of said temporary storage means may be selectively read outand selectively fed back into said spatial modulating means for furtherprocessing by the MLOENN as input signals to a next layer of the MLOENN.2. A multi-layer opto-electronic neural network (MLOENN) patternclassification apparatus as set forth in claim 1 wherein said spatialmodulating means is geometrically partitioned to permit the independentstorage of individual MLOENN layer weight vectors, ν.sup.(k,i (i=1,2, .. . , N-1; k=1,2, . . . , K), and the independent excitation ofindividual MLOENN layers by patterns σ.sup.(i) (i=1,2, . . . , N-1) suchthat each ν.sup.(k,i) and σ.sup.(i) occupy a same pixel of said spatialmodulating means.
 3. A multi-layer opto-electronic neural network(MLOENN) pattern classification apparatus as set forth in claim 1wherein said spatial modulating means further includes means for phaseencoding the spatially modulated spatially uniform laser beam inaccordance with a random, two-dimensional phase encoding function.
 4. Amulti-layer opto-electronic neural network (MLOENN) patternclassification apparatus as set forth in claim 1 wherein said detectingmeans includes a linear array of photoresponsive elements disposedwithin and along a correlation plane of said second Fourier transformlens.
 5. A multi-layer opto-electronic neural network (MLOENN) patternclassification apparatus as set forth in claim 1 wherein said nonlinearprocessing means has a sigmoidal transfer function with adjustablethreshold and saturation points.
 6. A multi-layer opto-electronic neuralnetwork (MLOENN) pattern classification apparatus as set forth in claim5 wherein said nonlinear processing means is either an electroniclook-up table or an electronic saturable amplifier.
 7. A multi-layeropto-electronic neural network (MLOENN) pattern classification apparatusas set forth in claim 1 wherein said temporary storage means is adigital electronic memory.
 8. A multi-layer opto-electronic neuralnetwork (MLOENN) pattern classification apparatus as set forth in claim1 wherein said volume holographic medium provides spatial filtering,within the correlation plane, of an electric field distribution, thespatial filtering being provided in accordance with the Braggselectivity of said volume holographic medium.
 9. A multi-layeropto-electronic neural network (MLOENN) pattern classification apparatusas set forth in claim 1 wherein said volume holographic medium iscomprised of a material selected from the group consisting of iron-dopedlithium niobate (LiNbO₃ :Fe), strontium barium niobate (SrBaNbO₃),photorefractive photopolymers, and photochemical photopolymers.
 10. Amulti-layer opto-electronic neural network (MLOENN) patternclassification apparatus as set forth in claim 1 wherein said spatiallight modulating means is comprised of means selected from the groupconsisting of ferroelectric liquid crystal devices, twisted nematicliquid crystal devices, deformable mirror devices, and magneto-opticdevices.
 11. A multi-layer opto-electronic neural network (MLOENN)pattern classification apparatus as set forth in claim 1 wherein saiddetecting means includes a two-dimensional array of photoresponsiveelements.
 12. A method for storing (K) Fourier-space holograms within avolume holographic medium, comprising the steps of, for K Fourier-spaceholograms ν.sup.(k,i) (k=1,2, . . . , N):loading a weight vectorν.sup.(k,n) into a spatial light modulator means using a predeterminedand fixed lexicographic ordering scheme; spatially modulating a firstplane-wave laser beam in accordance with the loaded weight vector;multiplying a light pattern transmitted by the spatial light modulatormeans in accordance with a random, two-dimensional phase encodingfunction; applying the phase-encoded light pattern, with a first Fouriertransform lens means, to approximately a midpoint of the volumeholographic medium so as to generate, at the midpoint, a Fouriertransform of the phase-encoded light pattern while simultaneouslyapplying a second plane-wave laser beam, which is temporally coherentwith the first plane-wave laser beam, so as to illuminate the medium atan angle ψ_(k) to a z₂ -axis of the medium and within a x_(2-z) ₂ plane;forming a weight vector hologram ν.sup.(k) within the medium; changingthe angle ψ_(k) to an angle ψ.sub.(k+1) ; and repeating the precedingloading, spatially modulating, multiplying, applying, forming, andchanging steps for each k=1,2, . . . , K until all ν.sup.(k) have beenstored in the medium.
 13. A method as set forth in claim 12 andincluding a step of fixing the weight vector holograms within themedium, the medium being formed of LiNbO₃ :Fe, the holograms being fixedby heating the medium to a temperature of approximately 130° C. for aperiod of approximately 25 minutes.
 14. A method for generating vectorinner products (ƒ_(k).sup.(i)) for accomplishing pattern recognition,comprising the steps of:loading an unknown pattern vector σ.sup.(i)(i=1,2, . . . , N) into a spatial light modulator means using a samepre-determined and fixed lexicographic ordering scheme that wasinitially employed to store a weight vector ν.sup.(k,i) (k=1,2, . . . ,K; i=1,2, . . . , N) template into a volume holographic medium;spatially modulating a first plane-wave laser beam in accordance withthe unknown pattern vector; employing a phase encoder means to multiplythe spatially modulated first plane-wave laser beam by a random,two-dimensional phase encoding function; transmitting a phase encodedlight pattern representative of the multiplied spatially modulated firstplane-wave laser beam from the phase encoder means to a first Fouriertransform lens means; generating within the volume holographic medium,with the first Fourier transform lens means, a Fourier transform of thephase encoded light pattern; generating, within the volume holographicmedium, an angular spectrum of plane waves having amplitudesproportional to vector inner products σ.sup.(i) •ν.sup.(k,i) (k=1,2, . .. , K; i=1,2, . . . , N) and propagation angles of ψ_(k) (k=1,2, . . . ,K) which correspond to reference angles ψ_(k) of a plane-wave coherentoptical signal; focusing onto a detector means, with a second Fouriertransform lens means, plane waves of the angular spectrum of plane wavesgenerated within the volume holographic medium; reading out, from thedetector means, signals representing vector inner products correspondingto the plane waves focused upon the detector means; serially andnonlinearly processing the output of the detector means; temporarilystoring the nonlinearly processed output of the detector means; feedingthe temporarily stored output of the detector means back into thespatial modulating means for further neural network processing as inputsignals of a next pattern vector σ.sup.(i+1) and reading out thetemporarily stored output of the detector means as pattern recognitionresults.